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Two projects that have explored the feasibility of 
computer-generated verbal score reports for the Iowa Tests of Basic 
Skills are reported. The firsts the Madison Project^ designed^ 
produced, and evaluated computer-generated testing reports that were 
useful to teachers and Informative to parents. The reporte^ which 
were narrative in format^ were structured as follows: the first five 
paragraphs described the pupil's performance on the Iowa Tests of 
Basic Skills as a whole and on each of the sub-tests; then, lists of 
specific areas were printed for which the pupil was diagnosed as 
either proficient or deficient. The teacher report was very similar 
to the parent report with the addition of imbedded percentile ranks. 
A class-summary report for the teacher was also generated, This^ 
report discussed the class as a whole and concluded with specific 
suggestions to the teacher for improving class perf oriiiance in which 
their performance was weakest. Narrative testing reports were 
generated for a random half of 52 fourth-grade classes in 17 schools. 
The greatest shortconiing of these reports was the absence of a pupil 
report. In the second project. The lowa Project, which developed a 
model for a n&rrative reporting system for the fifth-grade level Iowa 
Tests of Basic Skills^ the most important changes from the Madison 
Project werei (1) the addition of a pupil report; (2) the inclusion 
of growth statements; and (3) the use of three alternative levels of 
the parent and pupil reyorts^-^levels that are based on overall pupil 
performance on the test battery* Examples of the parent and teacher 
reports from the two projects are provided. (DB) 
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The use of standardised tests in the schools of America is under 
serious fire. (See Black, 1963| Goslin. 1963. 1965, 1967; Gross, 1962| 
Hoffmann, 19621 McCarthy. 1968j Mahler and Sn^Uanburg, 1963; Ohlesen. 
1963.) Parents complain that test results are used to label children 
and to restrict their educational oppdrtunities . Teachers complain that 
^ test results are misused to Judge teaching affectivenass . Psychologists 

say that tests are responsible for Inadaquata self=concepts held by 
^ children. Critics have sometimes raised questions about the relevance 

^ 2 '^^"8 measured by the testa.' but generally, the criticisms 

CS . °" '^"^"i" being used rather than on the 

Q tas=s themselves. It seems to me highly significant that at the same 

time thau the NBA was calling for a muratorlu. on standardised testing 
in the ochools, it was also calling on superintendents and boards of 
education to refuse to hire the s^duates of teacher education programs 
if those programs failed to include instruction in the interpretation 
and use of teats. The call for a moratorium, then, appears to reflect 
a recognition that tests are professional tools and a belief that the 
profeaalon is not at present adequately prepared to make proper use of 
the test results they are getting. 
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Ideally, when you seek medical advice^ you expect that the phyaician 
will administer appropriate tests to determine how various bodily systems 
are functioning; and although you may know something about the numerical 
systems in which the test results are recorded, you will generally 
expect the physician to report^ not the actual test scores, but verbal 
statements telling you what the numbers mean. The busy physician 
doesn't always succeed in achieving the ideal. Similarly, the ideal 
use of standardized tests in the educational setting would include an 
opportunity for all concerned individuals to have the results of testing 
asqplained in meaningful terms by somebody who has sufficient technical 
background to understand what the tast iicores means sufficient verbal 
skill to make the necessary translation p and sufficient time to do the 
job* Sometimes 5 particularly if teachers and administrators are 
unusually well trained and if the number of pupils to be served is 
small J practice approaching the ideal can be observed. Pupils, parents, 
teachers, school administrators, members of boards of education, and 
even the genaral public facing a vote on support for the schools - all 
are provided appropriate and meaningful interpretations of test results. 
Generally, however, there just aren't enough professional interpreters 
to get the job done on an individuallied basis. Some new way must be 
found for dealing with the problem. = 

The electronic computer has proved highly useful in the acoring 
and reporting of test results in numerical form* But computer output 
is not limited to numerical output alone | if properly programmed, the 
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cQmputer i.^j capable of generating reports in narrative format* In 
this paper w^. report two projects that have eKplored the feasibility 
of computer-generated verbal score reports for the Iowa Tests of Basic 
Skills* The firsts which we will refer to as the Madison Project ^ 
might be considered a pilot study involving, on a small scale j all the 
steps in the development and use of a verbal reporting system. The 
second p the Iowa Project, has eoncentrated on the development of a 
comprehensive reporting models that is, a detailed plan for turning 
numerical reports into verbal reports that will communicate clearly and 
accurately to the intended audience. Unless this can be accon^lished j 
it doesn't make much sense to undertake the e^^ensive and time^con"- 
suming task of building the reporting system itself. 

THE MADISON PROJECT 

The purpose of the Madison Project ( Ma thews j 1972) was to designp 
produce, and evaluate CQmputer^generated testing reports that would be 
useful to teachers and informative to parents , without requiring either 
parents or teachers to spend a lot of time preparing the reports or 
learning how to interpret them. In both cases j the reports were in a 
narrative format. A unique report was generated by a computer for 
each pupil 5 based on his scores on the fourth-grade level of the Iowa 
Tests n£ Basic Skills. 

Attempts were madei to increase the eonmiunicative role of the re- 
ports by putting them in a narrative format | to personaliEe the reports 
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by lacing each report with the pupil's name and the appropriate personal 
pronouns, to make the reports more understandable by aliminatlng or 
deamphasizing certain interpretative concepts (viz. grade-equivalent 
s. ea and percentile ranks) that sometimes are not fully understood; 
to make the reports more informative .by indicating possible proficiencies 
and deficiencies I to make the reports more useful to teachers by 
generating suggestions for change and sources for assistance; to make 
the reports more valuable to parents by giving them a permanent copy. 

Verbal descriptors were chosen to correspond to different ranges 
of percentile ranks, and a basic form for the explanatory paragraphs 
was developed, and then several parallel forms of each paragraph were 
written to provide variety of style. The first five paragraphs of 
each report described the pupil's performance on the test battery as 
a whole and on each of the sub=tests. Next, lists of specific areas 
were printed for which the pupil was diagnosed as either proficient or 
deficient— areas, in which he was neither, were suppressed. These 
decisions were based on Item analyses performed by the teachers In the 
Madison Cwisconsln) Public Schools and criterion levels set by them. 

Figure 1 about here 
Figure 1 is a sample of a report to parents. The teacher report 
was very similar to this report, with the addition of Imbedded per- 
candle ranks, The parent reports were distributed at a parent-teacher 
conference and were the basis for a discussion of the child's performance. 

One other report was generated— a class-sunmiary report for the 
^®^^her. This report was similar In format to the report on an . - . 
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individual student, but it discuased the class as a whole and concluded 
with specific suggestlona that might be useful to the teachar in im- 
proving class performance in the tested areas in which their performance 
was weakest. 

Narrative testing reports were generated for a random half of fifty- 
two fourth-grade classes in seventeen schools. Three major conclusions 
were reached as a result of the evaluation of the narrative reports 
when compared to the traditional reports! (1) The teachers preferred 
rtceivlng the narrative reports to the traditional reports; (2) the 
teachbra who received the narrative reports felt better prepared to 
communicate Information obtained from the tests to the parents of their 
pupils than did teachers who received the traditional reports | (3) 
receipt of the narrative reports had a greater impact on the teachers' 
perceptions of pupil performarce than did receipt of the traditional 
reports. 

The Madison Project did a respectable job at what it did, but It 
did not do enough. It took a major step in humanizing testing reports ■ 
and in offering an alternative to the obstacle of terminology and 
numerical ratings, but it only reported the pupil's performance comparod 
to other fourth graders—no reference was mada to the child's per- 
formance compared to his own previous performance. The Madison reports 
generated the same basic format for all chlldran, making no allowances 
for different needs at different levels of performance. The greatest 
shortcoming of these reports, however, was the absence of a pupil report. 
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THE IOTA PROJECT 

A project conducted during the summer of 1972 under the auspices of 
the Iowa Testing Programs developed a model for a narrative reporting 
system for the fifth-grade level Iowa Tests of Basic Skills. The model 
built upon the Madison Project , but the emphasis was on the basic con- 
ceptualisation rather than on developing the tachnology for in^lementing 
the models 

The most important changes from the Madisnn Project to the Iowa 
Projact were: (1) the addition of a pupil report; (2) the inclusion of 
growth statements; and (3) the use of three alternative levels of the 
parent and pupil reports—levels that are based on overall pupil 
performance on the test battary. An exampla of the kind of pupil 
report we developed appears as Figure. 2, 

Figure 2 about here 

The growth statements are verbal in nature and are basad on the 
dlfferenca between the score obtained by the pupil on the current year's 
testing and the score obtained at some previous administration , usually 
one year earlier* The three alternative levels of the parent and pupil 
report approach their audiences from different bases i a low performing 
pupil is given encouragement after showing growth ^ even though his 
performance level is low, while a high^performing pupil may be given 
specific suggestions on how to improve in an area in which his per- 
formance is highj if it is his weakest area. 

Other changes also were designed i the parent report (see Figure 3) 
was eKpanded to include a general explanation of tests and test results | 
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suggestions for increasing performflnce were included in the parent and 
pupil reports; comments to reinforce growth ware added | the reading 
level of the reports was lowered j the teacher report was shrunk to a 
skeletal report (see Figure 4). 



Fisures 3 and 4 _about here 

CONCLUSION ' 

There is no doubt that testing reports in narrative format can be 
prepared using computer technology; the basic question is whether or 
not such reports can be made sufficiently accurate and informative to 
warrant the expense of developing an operating system. 

The Madison project showed that, at least from the viewpoint of 
parents and teachers using the yeports, even a very simple system does 
seem to facilitate communication. The Iowa Tasting Programs, summer 
project has demonstrated that a much more sophisticated model can be 
developed. The next steps are to see whether or not this more complex 
model will turn out to provide meaningful information to pupils, parents, 
and teachers and whether or not it is economically feasible to imple- 
ment the model on a large scale. 
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PARENT REPORT ON INDIVIOUAL PERFORMANCi ON THE IOWA TESTS OF iASIC SKILLS 

ADMlNliTiREDSEPTBMBER 1970 
MADISON PUQLIC SCHOOLS 
MADISON, WISCONSIN 

STUOfNT: DANIEL MARSHALL 
TEACHER: MRS,ANaSTASI 

ACCORDING TO THIS TEST, DANliL'S OVERALL ACHIEVEMENT LEVEL IS AT THI 75 Pi RCiNTlLE 
WHEN COMPARED WITH STUDENTS IN GRADE 4 IN THE MADISON SCHOOL SVSTf M AND AT THf 7i 
PiRCiNTlLR WHEN COMPARfD WITH A NATIONAL SAMPLE OF GRADE 4 STUDENTS. 

THi TEST PROVlDiS INFORMATION IN FIVE MAIN AREAS: DANIEL'S VOCABULARY SCORE IS NEAR 
AVERAGE, IN THE AREA OF LANGUAGE SKILLS HIS SCORE !i QUITE STRONG. AND HIS SCORE IN WORK- 
STUDY SKILLS IS VERY STRONG. HIS ARITHMETIC SCORl IS EXCiPTIONAL. 

THi TiST LOOKED AT FOUR SPECIFIC LANGUAGE SKILLS. AND THIS STUDENT WAS RATf D AS 
FOLLOWS: QUITE STRONG IN SPELLING AND QUITE STRONG IN CAPITALIZATION. RATHER WiAK IN 
PUNCTUATION AND NEAR AVERAGilN WORD USAGE, 

n.M.Jf'el^^ ^^^^ °^ ^OR»<"STUDYSKILLS.THRiESPiCl>ICSKlLLSWiRETES 

DANIiL SEEMS TO BE QUITE STRONG WHILE HE IS NEAR AVERAGE IN READING GRAPHS AND TABLES IN 
HIS KNOWLEDGE AND USE OF REFERENCE MATERIALS Hi is QUITE STRONG. ' 

•'^■^^^e AREA OF ARITHMITIC, HE APPiARS TO ii NEAR AVERAGE IN CONCEPTS AND VERY 
N PROBLEM SOLVING WHILE Hi TENDS TO BE iXCiPTlQNAL IN THE MODERN MATHiMATICS SUPPLEMENTARY 

DIAGNOSTIC KEYS INDICATE THAT DANIEL IS PROFICIENT IN; 

SOLVING PROiLEMS INVOLVING MONEY. 

SUBTRACTION, 

ADDITION. 

UNDERSTANDING THE CONCEPT OF WHOLE NUMBER IN MODERN MATH. 
DIAGNOSTIC ALSO INDICATi THAT DANIEL IS DEFICIENT IN: 

SPgLLlNG lECAUSE OF PROBLEMS IN VOWEL SUBSTITUTION 
USING THE COMMA, 

PUNCTUATION lECAUSi OF OVERPUNCTUATION, 

THE ^ur^^^^"^ iVALUATiO IN SiViRAL WAYS^ BY THE TEACHIR. BY THE PARENTS OF 
THE S I UDENT AND, OF COURSi, BY THi STUDENT HIMSELF. TEST RESULTS DO NOT REPLACE ANY OF THESE 
sS ^O i^ROR^'' ^""^^ ^''^ • OATA Ai^S TiST. UKf ^^^^ 



COMMiNTS ABOUT DANIEL MARSHALL FROM MRS. ANASTASh 
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PUPIL REPORT ON PERPORiaNCE ON THE IOWA TESTS OF BASIC SKILLS 
Administered January 1973 
Iowa City Public Schools 
Iowa City t Iowa 

HORACE mm SCHOOL 

Pupil I JOHN BLACK GRADE 5 Teacher I MRS, RICfi^RDSON 

A few weeks agOp you took the Iowa Teits of Bailc Skills, You may remember 
that it had questions on vocabulary, spelling* capitalization, punctuation, and 
reading maps, graphs and tables. There were also questions about short stories 
you w^re asked to read, and there were problems In arithmetic, too. 

These teats give you a chance to find out how you are*' doing in the skills 
that are used in school. Your teacher gave the teats to everyone in che class 
to help her see where the class needs most workt Another reason for giving the 
testa was to give you information on your skills— which skills are strong p and 
which onea are weak* It's good to find out which ikills need attention, because . 
then you can work to improve them. 

This report will tell you how you did In the tests* The first coronent for 
each test tells you how strong or weak your score was in that test. Your score 
is compared to the scores of other students in siKth grade in Iowa. Your score 
is also compared to the score you had on a similar test you took in the fourth 
grade t This shows how much you have grown in that area since last year. 

In the test on vocabulary, your score was near average. Your 
growth during the past year was below average* One good way to 
li^rove your vocabulary would be to tak^ a little time to look in 
the dictionary for the meaning of words you are not sure of. It 
takes extra timei and sometimes you might have to look up the 
same word several times before you remember its meaning* But your 
vocabulary will get bigger and you will better understand what you 
read and hear. And you will be better able to express what you 
think . 



Your performance on this teat was stro^. You showed normal 
growth in reading in the past year. In this test you did very 
well in seeing and understanding lmporf:ant facts and detalli in 
what you read. 

On the four tests that measured the ma^anics of wrltlngi 
your overall performance was near average. Tour growth from 
last year was below average • These are some areas where a little 
sxtra work would pay off. 

SPELLING-^Your score on this teat was very weak. You showed mini" 
mum growth since last year. It appears that you sometimes are not 



VOCABULARY 



FADING 
COMPREHENSION 



L^GUAGE 
SKILLS 
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WORK-STUDY 
SKILLS 



SKILLS 



sure which vowels correccly spell a word. If you really want to 
in^rova your ipelling, you canl U«e a dictionary to chock" 
Your teacher will give you help too, if you want it 



spellings 



C^rTALlZATION--In capicall.atlon your performance was near average. 
You showed normal growth* a • 

PUNCTUATION=^Your performance in this area of language .kills was 
less than average. Your growth was below average. You tended to 
use coo much punctuation* 



USAGl-.=Your performance in the proper use of words was near average. 
Your growth since last year was normal. In this 
well ill using verbs. 



test you did very 



There were three tests that measure work-study skills- a 
map reading test, a test of reading graphs and tables, and a test 
of your knowledge and use of reference materials. In general 
your performance was near average. You showed normal growth.* 

MAP READING--Your score on this test was near average. Your 
growth since last year was below average. You seemed to have 
trouble locating places on maps and globes. Some practice in 
using a map of Iowa City and the state of rowa would be helpful. 

f« l") ^' ^ ^^^^ ^ "^P " ""^^d 

READING GRAPHS AND TABLES~In this test your score was stroni . 
Your growth in this area was above average. You've improved In 
this area and should be proud of It, 

REFERENCE MATERIALS^-Your knowledge and use of reference materials 
18 near average, according to the test. You've shown normal growth 
since last year. You did miss questions on the use of an index. 

Your overall performance in the two testa of arithmetic skills 
shows that you are strong In this area. Your growth in these skills 
since last year was above average. Keep it i^f se sfcij.is 

tttTHmiATlCS CONCEPTS-Your score on this test was strong 
showed above average growth in this area.. 



You 



^HHMATICS PROBLm SOLVING-=You have shown atrong performance in 
this area. Your growth In this skill was above average. You did 
well in solving problems involving money. 

m.r>,J^.f""w^i'^f^ testa shew that you have done very well in improving your 
?au .n' ''''' ^"'^ in voc^ulary and langLgl 

skills. You will want to pay close attention to spelling anJ punctuation You 
also, showed good progress in reading graphs and tables. P""«"atlon. You 



Figure 2 (cont j 



Of all the five test areas , you did best In mathematics. You need to Improve 
the most in work-study skills, especially inap reading and ualng reference materialg. 
You could also Iniprove your language skllli, mainly spelling. 

... . i'. . ■ . ■ . ■ . . .... .. " 

There are other thingi you do at school that are also l^ortant. You are 
growing in many ways, not only in your .ikills, but also as a pereon. The Iowa . 
Testa of Basic Skills do not measure these other areas of growth, so they have not 
been talked about here* It's important that yoUj your parents, and your teacher 
talk about these other areas'^^your everyday activities in all areas of achool, 
your Interests, and how you feel about school. That way, all of you can be more 
sure that you are getting the most out of your es^erlence In school. 
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PARENT REPORT ON PUPIL PERFOmKCE ON THE IOWA TESTS OF BASIC SKILLS 

Administered January 1973 
Iowa City Public Schools 
Iowa City^ Iowa 



Pupil- JOHN BLACK 



HORACE MANN SCHOOL 
GRADE S 



Teacher : MRS . RICHAIU)SON 



_ John's achoQl experience ii evaluated In many ways and by various Deool^- 

performance last year. Remember, however, that the lova Tests of B„eL skills 
teU Z ''-^'^ ""-^ performance" No'lesf c'li" 

.ion Sou^th^fpS:^^^^^^ °^ ^"^o.„.= 

Second, a growth statement^-his performance this year la compLed to his 
formonce laat year in order to see his growth In, each skill. ^ 

VOCAnULARY ^John's performance on this test was near average compared 

to the performance of other fifth grade pupils !„ Iowa'. He 
. showed below average growth In this area since last year. You 

i^ght encourage him CO look in a dictionary for word meani^^^ 
whenever ha is uncertain. minings 



READING 
COMPREHENSION 



On this 



, ' i * performance was strong. His scores 
showed normal growth in this area in the past year. He did 

''nF/^ understanding Important facts 

and details in what he read. 



LANGUAGE 
SKILLS 



J"" =e"""d John's skills in the mechanics of 
i^ritini. His overall performance in language skills was near 
average His growth from last year was lellw average! En- 
courage John to take the time to describe fully an event he has 
experienced. Ha may then be encouraged to put it In writing 
and mall It to a friend or relative. 

SPELLlNQ--John'8 performance in the spelling test was very weak. 
He showed very little growth ginee last year. He tends not to 
notice wrong vowels in misspelled words. 



Figure 3 (cont. ) 



WORK-STUDY 
SKILLS 



MATHH^ATICS 
SKILLS 



p P 18 not sure of the rules of punctuation. 
USAGE— In the teat nf f.h= 

verbs, and adJe"LL) John !' ««nPl^. nouns, 

the symbols. used on the Mp° distances end to Interpret 

•bov. average. ^" "^^ growth In this area was 



area, it would b. gold If j"i 5 °" average in this 
Z^^ ^S^ ?-T2aJSg a^^^'tL^ni--^ 

•■ ."na%'«%Er"- that hU .athetotics 

above avarage grS on'tSls •>•> "ho™ 

oal problem was strong He ^fS ,! "! *" "1^'1"8 "athematl- 

probla. splvlng. -Je dld well In .o?w 8"""- »" 

»=n,y. He did well 1„ subt"cJjon? ^ '■""oMm 

v.r,«n..pe,t..arLH'^S--^El^^^^^^ 



Figure 3 (cont.) 



and language ikills, particularly spelling and punctuation • He showed good 
progreaa in reading graphs and tables. 

Now let's aummar lie John's performance on this test this year. It can be 
said that he did beat in mathematics and he needs improvement the most in his 
work-^study skills (especially map reading and using reference ma CGrials) . He 
also Qould improve his language skills (particularly spelling)* 

This Inforroation used In conjunction with obier/ations provided by you, 
John's teacher i and John should result in better meeting the goal you all share 
helping John make the most of his experience In' school* 

In order to make this report as accurate and up^to-^date as possible, Mrs, 
Richardson has added her comments below. 



Figure 4 



TEACHER REPORT ON INDIVIDUAL PEM'ORMANCE OH THE IOWA TESTS OP BMIC SKILLS 
^ Mmlnlsterad January 1973 ' 

Iowa City Public Schooli 
Iwa City, Iowa 



Pupil- JOHN BIACK 



HOr^E SCHOOL 



Teachar I MRS , RICHARDSON 



The first part of each coronet raferg to JohnU tast performanea this year, (The par 
centilaa show his performance compared to other pupili in Iowa.) Tha growth BtaCements 
rafar to John's test parformance thig year eompared to last yearns testing. 



VOCABULARY 



Naar average perforraance (61%lle) and below a veraga growth. 



REDING COMPREHENSION 



Strong (71%ile) with normal growth. John did very well in 
reeognlilng and understanding important facts and details 
in raading. 



LANGUAGE SKILU 
Spelling 

CapltallEation 
Punctuation 

Usage 



Near flveraga performance (45%ile) with below avaraga growth. 

Heak (28%ile) with minimal growth. Vowel iubstitution is an 
area in which John needs help. 

Naar avarage performance (48^,ile) with normal growth. 

Near average perfprmance (40%lle) with less than avarage growth, 
John tends to use too much punctuation. 

Near average per fprmance(62%lle) with normal growth. He did 
very well in using verbs. 



WORK-STUDY SKlIiS 
Map Reading 

Graphs^Tables 
Refarence Materials 



Near averaga performance <44%lle) with normal growth. 

Near averaga, perfoTOance <34%ile) with below averaga growth. 
Locating places on maps and globis in a problem for John. 

Strong (73%lle) with above average growth « 

Near average per£omance (40%lle) with normal growth. Use of 
an Index is an area in which he needs help. 



mTHayiTICS SKILLS 
Mathematics Concepts 
Problem Solving 



Strong (76%lle). with above average growth. 

Strong (72%lle) with above average growth. 

Strong (79%lle) with growth that was above average. He did 
wall in solving problems Involvin^^^^^^^ He did well in 

subtraction. 



